Moving from polling to long-polling

If you’ve built a REST API that clients poll for updates, you’ve probably considered adding a realtime push mechanism. Maybe you’ve been putting it off due to the added complexity, or the impact it might have on your API contract. These are valid concerns, but push doesn’t have to be that complicated.

In this article we’ll discuss how to update an API to use long-polling. It assumes:

1. You have an existing REST API.
2. You have clients repeatedly polling this API.

Long-polling is not the same as “plain” polling. With long-polling, the server delays the response to the client if there is no new data yet. This enables the server to respond instantly whenever the data does change. Aside from providing actual realtime updates, what’s great about long-polling is that technically it’s still RESTful, requiring hardly any changes to your API contract or client code.

Of course, long-polling may not be as efficient as streaming mechanisms like Server-Sent Events or WebSockets, but it’s inarguably more efficient than plain-polling. Let’s compare:

Long-polling is a great way to dip your feet in the realtime waters without having to dramatically change your API contract and client code.

API contract

Alright, so what kind of destruction do we need to do to your API contract? Hopefully not much at all.

Fundamentally, you need a way for the client to make a conditional request for data. If your API supports ETag and If-None-Match then you’re already there. Queries against a timestamp are usually not good enough, as data could change twice with the same timestamp. You’ll want something more precise like a resource hash or version.

Assuming your API has a good way to handle conditional requests, now you need to decide how the client should ask the server to delay a response.

It might be tempting to make the server automatically delay all requests that have failed conditionals. Then your API contract wouldn’t need to change, and all existing plain-polling clients would magically become long-polling clients! However, this may break clients that depend on conditional checks responding quickly. We recommend making the long-polling behavior opt-in to be safe.

As for how to do the opt-in, we suggest using the Prefer header. The Prefer header is already an RFC, and one of the things you can do with it is tell the server that you are willing to wait for a response up to a certain amount of time. For example:

GET /resource HTTP/1.1
Host: example.com
If-None-Match: "etag-of-resource"
Prefer: wait=120

This is saying the client wants a response from the server within 2 minutes. Your API could use this as a hint that it’s acceptable to delay the response.

Updating your API documentation is easy, just add an explanation of the Prefer header.

Implementation, client side

On the client side, simply update any polling requests to ask for long-polling. The client code otherwise stays the same. For example, if you’ve got polling code in Python that looks like this:

import time
import requests

etag = None
while True:
    headers = {}
    if etag:
        headers['If-None-Match'] = etag
    resp = requests.get('http://example.com/resource', headers=headers)
    if resp.status_code == 200:
        etag = resp.headers.get('ETag')
        process_response(resp.json())
    elif resp.status_code != 304:
        # back off if the server is throwing errors
        time.sleep(60)
        continue
    time.sleep(5)

All you need to do is modify a couple lines of code:

import time
import requests

etag = None
while True:
    headers = {'Prefer': 'wait=120'}  # <----- add hint
    if etag:
        headers['If-None-Match'] = etag
    resp = requests.get('http://example.com/resource', headers=headers)
    if resp.status_code == 200:
        etag = resp.headers.get('ETag')
        process_response(resp.json())
    elif resp.status_code != 304:
        # back off if the server is throwing errors
        time.sleep(60)
        continue
    time.sleep(0.1)                   # <----- reduce delay between requests

Hopefully modifying your actual client code projects is just as easy. It might take a little more effort if your polling isn’t using conditional requests or a backoff strategy yet, but that shouldn’t be too hard to fix.

Implementation, server side

The real work is on the server side. How you go about supporting long-polling greatly depends on your programming language and/or server stack.

Here we’ll describe how to implement using Pushpin, our open source proxy server that makes building realtime APIs easy.

First, install Pushpin and configure it to forward traffic to your backend.

Next, when the backend receives a request containing a conditional that fails, respond with a “no data” response along with instructional headers. For example, here’s how a “no data” response might look for an If-None-Match conditional request:

HTTP/1.1 200 OK
ETag: "etag-of-resource"
Grip-Status: 304
Grip-Hold: response
Grip-Channel: /resource; prev-id=etag-of-resource
Grip-Timeout: 120

Pushpin removes the instructional Grip-* headers from the response and changes the status code from 200 to 304, but it doesn’t forward it to the client right away. Instead it waits up to 120 seconds before responding, unless it receives data on the /response channel.

If no data is received in time, Pushpin sends a responds to the client that looks like this:

HTTP/1.1 304 Not Modified
ETag: "etag-of-resource"

Lastly, whenever the resource changes, the backend publishes the resource’s new value to Pushpin’s private control API:

POST /publish/ HTTP/1.1
Host: localhost:5561
Content-Type: application/json

{
  "items": [
    {
      "channel": "/resource",
      "id": "new-etag-of-resource",
      "prev-id": "previous-etag-of-resource",
      "formats": {
        "http-response": {
          "code": 200,
          "headers": [
            [
              "Content-Type",
              "text/plain"
            ]
          ],
          "body": "The content of the resource."
        }
      }
    }
  ]
}

This will cause Pushpin to send a response to the client that looks like this:

HTTP/1.1 200 OK
ETag: "new-etag-of-resource"
Content-Type: text/plain

The content of the resource.

Conclusion

Realtime push doesn’t have to be complicated. If you’re already doing plain-polling with your API, switching to long-polling can be easy with the right tools and it barely affects your API contract and client code. There’s really nothing to lose.